Real-time Image Processing System Research Articles

Single-metalens-assisted polarization imaging and edge detection for target recognition

Simultaneous capture of various light information, including polarization and edge information of the objects, has consistently been a fundamental concern within the field of target recognition. However, these tasks are typically accompanied by bulky optical components and active illumination methods, which significantly restricts their use in compact and lightweight applications. Here, we demonstrate a metalens-assisted imaging system that can simultaneously achieve polarization imaging and optoelectronic edge detection in a single shot with low consumption. The dielectric metalens is designed to achieve polarization imaging by dispersing the input polarized light into two orthogonal components, resulting in optoelectronic isotropic edge detection of two-dimensional images after digital post-processing. Compared with the algorithmic methods using a convolution kernel, the proposed system has a much lower computational complexity. The work presented in this study demonstrates the potential applications in machine vision and paves the way for the development of compact target recognition and real-time image processing systems.

Design of real-time digital image processing system based on high performance computing

Design of real-time digital image processing system based on high performance computing

Real Time FPGA Implementation of an Efficient High Speed Harris Corner Detection Algorithm Based on High-Level Synthesis

Computer vision systems use corner detection to identify features in an image. In applications such as motion detection, tracking, picture registration, and object recognition, corner detection is often one of the initial steps. In this paper, a real-time image processing system based on Harris corner detection was designed and implemented using Zynq architecture and model-based design tools. The system was based on a development board containing the Zynq-7000 chip, which consists of a combination of FPGA and microprocessor, and the image taken with a high-resolution camera was processed in real-time by applying color conversion and Harris corner detection. The filter hardware designs used in the system were made using the HDL Coder tool in Matlab/Simulink without writing HDL code. The hardware that receives images from the camera was designed on a model-based basis with the Xilinx Vivado 2020. The HDL code that was implemented on the Xilinx ZedBoard using Vivado software was then validated to ensure real-time operation with the incoming video stream. The results achieved exhibited superiority compared to prior implementations in terms of area efficiency (reduced number of gates on the target FPGA) and speed performance on an identical target card. Using the rapid prototyping approach, two alternative hardware accelerator designs were created using various high-level synthesis tools. This design used less than 50% of the host FPGA's logic resources and was at least 30% faster than current implementations.

Algorithms to Speed up Contour Tracing in Real Time Image Processing Systems

Contour tracing is an important pre-processing step in many image processing applications such as feature recognition, biomedical imaging, security and surveillance. As single processor architectures reach their performance limits, parallel processing architectures offer energy efficient and high performance solutions for real time applications. Parallel processing architectures, are thus used for several real time image processing applications. Among the several interconnection schemes available, Cayley graph based interconnections offer easy routing and symmetric implementation capabilities. For parallel processing systems with a Cayley graph based interconnection scheme, torus, we developed three accelerated algorithms corresponding to three existing families of contour tracing algorithms. We simulated these algorithms on a parallel processing framework to quantify the normalized speed-up possible in any torus connected parallel processing system. We also compared our best performing algorithm with the existing parallel processing implementations for Nvidia GPUs. We observed a speed-up of up to 468 times using our algorithms on a parallel processing architecture in comparison to the corresponding algorithm on a single processor architecture. We evaluated a speed up of 194 (and 47) compared to the existing parallel processing contour tracing implementation on Tesla K40c (and Quadro RTX 5000 GPU hardware respectively). We observe that for torus connected parallel processing architectures used for image processing, our algorithms can be used to speed up contour tracing, without any hardware modification.

EFFICIENT LINE DETECTION METHOD BASED ON 2D CONVOLUTION FILTER

The article proposes an efficient line detection method using a 2D convolution filter. The proposed method was compared with the Hough transform, the most popular method of straight lines detection. The developed method is suitable for local detection of straight lines with a slope from -45˚ to 45˚.  Also, it can be used for curve detection which shape is approximated with the short straight sections. The new method is characterized by a constant computational cost regardless of the number of set pixels. The convolution is performed using the logical conjunction and sum operations. Moreover, design of the developed filter and the method of filtration allows for parallelization. Due to constant computation cost, the new method is suitable for implementation in the hardware structure of real-time image processing systems.

A Manycore Vision Processor for Real-Time Smart Cameras.

Real-time image processing and computer vision systems are now in the mainstream of technologies enabling applications for cyber-physical systems, Internet of Things, augmented reality, and Industry 4.0. These applications bring the need for Smart Cameras for local real-time processing of images and videos. However, the massive amount of data to be processed within short deadlines cannot be handled by most commercial cameras. In this work, we show the design and implementation of a manycore vision processor architecture to be used in Smart Cameras. With massive parallelism exploration and application-specific characteristics, our architecture is composed of distributed processing elements and memories connected through a Network-on-Chip. The architecture was implemented as an FPGA overlay, focusing on optimized hardware utilization. The parameterized architecture was characterized by its hardware occupation, maximum operating frequency, and processing frame rate. Different configurations ranging from one to eighty-one processing elements were implemented and compared to several works from the literature. Using a System-on-Chip composed of an FPGA integrated into a general-purpose processor, we showcase the flexibility and efficiency of the hardware/software architecture. The results show that the proposed architecture successfully allies programmability and performance, being a suitable alternative for future Smart Cameras.

Image Processing Design and Algorithm Research Based on Cloud Computing

Image processing technology is a popular practical technology in the computer field and has important research value for signal information processing. This article is aimed at studying the design and algorithm of image processing under cloud computing technology. This paper proposes cloud computing technology and image processing algorithms for image data processing. Among them, the material structure and performance of the system can choose a verification algorithm to achieve the final operation. Moreover, let us start with the image editing features. This article isolates software and hardware that function rationally. On this basis, the structure of a real-time image processing system based on SOPC technology is built and the corresponding functional receiving unit is designed for real-time image storage, editing, and viewing. Studies have shown that the design of an image processing system based on cloud computing has increased the speed of image data processing by 14%. Compared with other algorithms, this image processing algorithm has great advantages in image compression and image restoration.

Special issue on deep learning for emerging embedded real-time image and video processing systems

Special issue on deep learning for emerging embedded real-time image and video processing systems

Alternatives to Bicubic Interpolation Considering FPGA Hardware Resource Consumption

Bicubic interpolation is widely used in real-time image processing systems because of its quality. The real-time implementation of bicubic interpolation requires a lot of hardware resources, especially the number of multipliers because it represents high computational complexity. In this article, a set of algorithms that approximate the bicubic interpolation and reduce the hardware resource consumption are proposed. The proposed algorithms are based on combining linear and cubic interpolations. These algorithms are surveyed and compared in terms of interpolation quality, number of adders, number of multipliers, adaptive logic modules, lookup tables (LUTs), registers, and maximum operating frequency. These algorithms are implemented and tested on an Intel Cyclone V target. This article provides various choices of interpolation algorithms to cater to different application requirements, including accuracy, hardware resource consumption, and throughput performance. The implementation codes are available at github.com/DreamIP/Interpolation.

Real-Time SAR Imaging Based on Reconfigurable Computing

Real-time synthetic aperture radar (SAR) image-processing systems on airborne or spaceborne platforms have constraints in terms of their power consumption, volume, and weight. In order to meet the requirements of these platforms and increase the flexibility of system application, this paper presents a SAR real-time imaging technology based on field-programmable gate array (FPGA) dynamic-reconfiguration technology. First, this technology is used to optimize the algorithm. The entire process flow is then classified into dynamic and static parts. The dynamic part implements numerical calculations, and the static part involves process control, data storage, and peripheral interface. Then, based on dynamic reconfiguration, the processing system is designed, including data interface, data storage, process control, and data operation. Finally, using flight data based on an unmanned aerial vehicle, the performance of the system is verified. The results indicate that the system needs to be run at a 200 MHz frequency for 85.9 s to process data with a size of 65536 ×65536. The system uses approximately 31.4% less resources and consumes 20% less power compared to the traditional method. These experimental results prove the effectiveness of the technology, laying the foundation for the design of high-performance real-time image-processing systems on airborne or spaceborne platforms.

FPGA-based architecture for bi-cubic interpolation: the best trade-off between precision and hardware resource consumption

An accurate and low-cost design of the image interpolation unit is a crucial part for many real-time image processing systems. To reach this goal, bi-cubic interpolation is generally selected because it provides the best trade-off between computational complexity and interpolation quality. The aim of this paper is to study the optimal hardware implementation of heterogeneous bi-cubic interpolation. Bi-cubic algorithm is reformulated and improved for FPGA implementation. This improved algorithm avoids twelve redundant calculations and reduces the number of multipliers by $$25\%$$ . Hardware precision versus resource utilization is studied to minimize the quantization error of hardware realization, and to obtain the best trade-off between design cost and accuracy. A compromise that reduces $$33,33\%$$ of  bit-width utilization with a precision higher than $$99.922\%$$ is reached. Besides, the proposed architecture is fully pipelined to reach high operating frequency. Instantiation on Xilinx and Intel targets shows the benefit of our approach, especially in terms of hardware resource consumption.

Automatic Thresholding Technique Using Reconfigurable Hardware

This research presents an efficient automatic thresholding technique based on Otsu’s method that can be used in edge detection algorithms and then applied as a plug-in for real-time image processing devices. The proposed thresholding technique uses an iterative clustering based method that targets a reduced number of operations. It is well known that the Otsu calculates the global threshold splitting the image into two classes, foreground and background, and choose the threshold that minimizes the interclass variance of the threshold black and white pixels. In this paper, a faster version of Otsu’s method is proposed knowing that the only pixels that have to be moved from one class to another class are the ones with values in between the previous two thresholds. This procedure yields the same set of thresholds as the original method but the redundant computation has been removed and, in this way, only few operations are required. The proposed thresholding technique has been implemented in software using C# programing language and in reconfigurable hardware on a Spartan 3E XC3S500E FPGA board using VHDL. The results obtained, presented for different digital images, confirm that the proposed iterative thresholding algorithm and architecture on FPGA can achieve the requirements to be included in real-time image processing systems.

Design of Real-time Image Processing System for Measurement and Calibration of PSP

The measurement of surface pressure in wind tunnel or in flight tests gives very important data for aerodynamics analysis. In fact, the pressure measurement gives valuable information about air flow phenomena such as shock, flow separation and boundary layer transition. In this paper a real-time image processing system for pressure sensitive paint (PSP) measurement and calibration has been designed using a calibrated pressure sensor. In the implementation of the system, TCL paper is used to luminaphore inside a windowed enclosure and a factory-calibrated pressure sensor for validation of measurement. The pressure measurement of PSP is based on the images acquired from a CCD camera sent to the MyRIO processor module in which the real-time image processing has been applied. The results are compared with the calibrated sensor from which the A and B coefficients of the Estern Volmer Equation have been calculated. During the several tests done for the pressure measurement, the accuracy of the PSP measurement has been calculated in this paper. Both normal and real-time image processing analysis have been compared as well.

Threshold optimization selection of fast multimedia image segmentation processing based on Labview

With the continuous improvement of computer technology information level, multimedia image processing technology is constantly updating and progressing, and it is more and more urgent to quickly perform multimedia image recognition processing. Multimedia image recognition is an important issue in image processing. Image segmentation is the basic premise for visual analysis and pattern recognition of multimedia images. The multimedia image recognition segmentation algorithm based on threshold selection is simple in calculation and has high computational efficiency, which makes it widely used in multimedia real-time image processing systems. However, due to the variety of threshold selection, it directly affects multimedia image segmentation effectiveness. In this paper, the research and discussion on some features of the multimedia image segmentation recognition algorithm based on threshold selection and its application are carried out. The segmentation effect of the maximum entropy method and the operation time of logarithmic entropy are studied. Then, the exponential entropy is used instead of the pair. The numerical entropy is improved, and the two-dimensional maximum entropy method is improved. Combined with the Otsu method, the information of the gray level of the 4 neighbourhood pixels is added. Experimental results show that the method used in this paper can effectively shorten the calculation time, highlight the edge features, and increase the threshold automatic selection accuracy and robustness.

Methodology for the implementation of real-time image processing systems using FPGAs and GPUs and their integration in EPICS using Nominal Device Support

There is an increasing interest to improve the processing capabilities for imaging diagnostics in fusion devices. Furthermore, the continuous upgrades of the hardware systems used in these tasks require a flexible platform that obtains the maximum performance of these technologies: cameras, frame-grabbers, and parallel processing architectures using FPGAs, GPUs, and multicore CPUs. This work presents a methodology for the development of real-time image processing applications using the ITER CODAC Core System (CCS) software tools and the hardware configuration defined in the ITER fast controllers hardware catalog, by means of Camera Link cameras, Camera Link FPGA-based frame-grabbers, and NVIDIA GPUs. The integration with EPICS is provided using the Nominal Device Support approach and the IRIO tools integrated into CCS. This device support provides complete control of the camera configuration parameters, distribution of the image processing functions between the FPGA and the GPU, and efficient movement of the data between the different elements of the architecture. To validate the full development cycle, an EPICS application has been developed implementing different algorithms frequently used in fusion applications such as histogram, detection of levels, and classification of zones within an image. Results of the performance obtained are presented highlighting the differences among various configurations running the selected algorithms.

Real-time image processing systems using fuzzy and rough sets techniques

Real-time image processing systems using fuzzy and rough sets techniques

Visual Impairment Simulator Based on the Hadamard Product

In this paper, a real-time image processing system designed to simulate visual impairment for the normally sighted is presented. The system consists of a video camera, a computer, and a virtual reality (VR) headset. Based on the Hadamard (or Schur) product of the camera's video signal and a set of predefined masks, users can experience eye diseases such as macular degeneration, diabetic retinopathy, glaucoma, hemianopsia, among others. A quantitative user study is presented to illustrate the most complex daily task people with visual impairments face: reading.

Development of a real-time lumbar ultrasound image processing system for epidural needle entry site localization.

A fully-automatic ultrasound image processing system that can determine the needle entry site for epidural anesthesia (EA) in real time is presented in this paper. Neither the knowledge of anesthetists nor additional hardware is required to operate the system, which firstly directs the anesthetists to reach the desired insertion region in the longitudinal view, i.e., lumbar level L3-L4, and then locates the ideal puncture site by instructing the anesthetists to rotate and slightly adjust the position of ultrasound probe. In order to implement these functions, modules including image processing, panorama stitching, feature extraction/selection, template matching and support vector machine (SVM) classification are incorporated in this system. Additionally, a user-friendly graphical user interface (GUI), which displays the processing results and guides anesthetists intuitively, is further designed to conceal the intricacy of algorithms. Feasibility and effectiveness of the proposed system has been evaluated through a set of realtime tests on 53 volunteers from a local hospital.

Evolutionary design of multiplierless lifting-based 2D DWT filters for low-resolution image processing

The most difficult challenge in developing a high-efficiency embedded image processing system relates to how to execute complicated Digital Signal Processing (DSP) algorithm with embedded processor, especially when processing high-resolution images. A whole system may start to run very slowly because of the large volumes of data being processed and the complexity of the algorithm, rendering it unable to achieve real-time processing. Executing motion detection in low-resolution image was already proved to be able to reduce the loading of data manipulation and to be able to help filter out noise and fake motion. However, smoothing filter pointed out in the work will affect the quality of image. Although the LL-band signal of a 2D Discrete Wavelet Transform (DWT) domain can retain the main part of the energy and information of the original image, a complex algorithm is more suitable for the application of a low-pass filter. Symmetric Mask-based DWT (SMDWT) has the advantage of reduced complexity, regular signal coding, and independent subband coding processing. In this paper, we use a Particle Swarm Optimization (PSO) approach to automatically evolve the multiplierless 2D SMDWT filters hardware architecture. The architecture employs only shift-and-addition operations to replace the complex floating-point multiplication and division operations. It can use shift-and-add based SMDWT filter to decompose a low-resolution image and do real-time image processing system design with low-resolution image processing technique.

Design and Hardware Implementation of Image Compression Denoising Based on Median Filter and Wavelet Transform

An image compression denoising method based on median filter and wavelet transform is proposed in order to overcoming shortcomings of traditional methods of image processing in this paper. This method combined hardware parallelism with software technology is enable to achieve image compression denoising and take into account algorithm validation, and fast response of the system. An real-time image processing system is design by this method. Design and hardware implementation of fast median filtering algorithm based on EP1C12 FPGA chip is realized and software simulation of median filter and wavelet transform is done. The experimental results show that this system has advantages of fast response characteristic, less time consuming and high signal to noise ratio.